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ABSTRACT 


A  method  for  direct  numerical  evaluation  of  the  cumulative  probability 
distribution  function  from  the  characteristic  function  in  terms  of  a  single 
integral  is  presented.  No  moment  evaluations  or  series  expansions  are  re¬ 
quired.  IntermeSiate  evaluation  of  the  probability  density  function  is  cir¬ 
cumvented.  The  method  takes  on  a  special  form  when  the  random  variables 
are  discrete. 
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NUMERICAL  EVALUATION  OF  CUMULATIVE  PROBABILITY 
DISTRIBUTION  FUNCTIONS  DIRECTLY  FROM 
CHARACTERISTIC  FUNCTIONS 


INTRODUCTION 

When  several  independent  random  variables  are  added,  the 
characteristic  function  of  the  sum  is  the  product  of  the  characteristic  functions 
of  the  individual  random  variables.  This  rule  holds  regardless  of  the  distri¬ 
butions  of  the  individual  random  variables,  and  whether  they  are  identically 
distributed  or  not.  Evaluation  of  the  cumulative  probability  distribution  of  the 
sum  variable  in  closed  form  is  often  very  tedious  or  impossible  to  achieve. 

This  is  especially  so  when  the  number  of  random  variables  added  is  large,  but 
not  large  enough  to  employ  the  Central  Limit  Theorem  with  accuracy. 

In  many  signal-detection  problems,  the  characteristic  function  of  the 
decision  variable  can  be  derived  in  closed  form  (or  evaluated  numerically 
fairly  easily).  Often,  however,  neither  the  probability  density  function  of  the 
decision  variable,  nor  its  integral,  the  cumulative  probability  distribution 
function,  can  be  obtained  in  closed  form.  Even  if  they  can,  they  are  frequently 
tedious  and  time-consuming  to  evaluate  (see,  for  example,  Marcum1).  In  this 
report,  we  present  a  technique  for  numerically  evaluating  cumulative  probability 
distribution  functions  directly  from  specified  characteristic  functions  in  terms 
of  a  single  integral.  Intermediate  evaluations  of  the  probability  density  func¬ 
tions  are  not  necessary,  and  no  moment  evaluations  or  series  expansions  are 
required.  The  technique  takes  on  a  special  form  when  the  decision  variable  is 
discrete. 

When  the  characteristic  function  of  the  decision  variable  (which  is  com¬ 
pared  with  a  threshold)  can  be  evaluated  for  both  the  signal-present  and  signal- 
absent  cases,  the  technique  can  be  applied  to  the  problem  of  obtaining  receiver 
operating  characteristics  (probability  of  detection  versus  probability  of  false 
alarm). 


ANALYSIS 

This  section  is  composed  of  two  subsections.  In  the  first,  a  general 
formula  for  direct  evaluation  of  the  cumulative  probability  distribution  function 
from  the  characteristic  function  is  derived;  in  the  second,  an  alternate  and 
more  useful  form  for  discrete  random  variables  is  presented. 
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GENERAL  DISTRIBUTIONS 


Let  random  variable 
characteristic  function  (CF) 


x  have  probability  density  function  (PDF)  p(x)  and 

f«>: 


f(£)  =  f  dx  exp  (i£x)  p(x), 
and 

p(x)  =2 7  J  exp  HU) 


(1) 

(2) 


(An  integral  without  limits  is  over  the  real  axis  from  -«  to  +« .) 


The  cumulative  distribution  function  (CDF)  Pr(Z)  is  defined  as  the 
probability  that  random  variable  x  is  less  than  or  equal  to  X : 


X+ 

Pr®  =  J  dxp(x). 


(3) 


The  upper  limit  means  that  an  impulse  in  PDF  p(x)  at  x  =X  is  to  be  included 
in  full.  It  will  be  convenient  to  define  the  modified  distribution  function  (MDF): 


■I: 


P®  =  /  dx  p(x), 


(4) 


where  an  impulse  in  p(x)  at  x  =  X  is  only  half  included.  At  points  of 
continuity  of  the  CDF,  Pr®  and  P®  are  equal.  At  a  point  of  discontinuity 
of  the  CDF,  the  MDF  P®  takes  on  a  value  halfway  between  the  limit  values 
on  either  side  of  the  discontinuity. 2  The  CDF  Pr®  can  be  obtained  from  the 
MDF  P®  via 


Pr®  =  lim  P®+*).  (5) 

•H)+ 

Therefore,  we  can  direct  our  effort  to  evaluating  either  the  CDF  Pr®  or  the 
MDF  P®,  depending  on  which  is  more  convenient. 
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When  Eq.  (2)  is  substituted  into  Eq.  (4),  we  notf  that  the  MDF 
becomes3 


-00 


df  exp  (-i£x)  f  (() 


dxexp(-iix) 


*T-1ir/Tf(t)exp('1*’ 


where  the  last  integral  is  a  principal  value  integral.  Since  the  PDF  p(x)  is 
real,  the  real  part  of  the  CF  f(£)  is  even,  and  the  imaginary  part  of  the  CF 
f(£)  is  odd;  i.e.,  f(-{)  =  f*(£).  This  allows  Eq.  (6)  to  be  manipulated  into  the 
forms 

00 

P(X)  =  f  T  Im{f(f)exp(-ifZ)} 

0 

0° 

=  J [lm|f«)}cos(a)- Re{f(s£)}sinOTJ  .  (7) 


Convergence  of  the  integrals f  at  the  origin  is  guaranteed  by  the  fact  that 


*£vi3  integrable  at  the  origin  if  No  momenta  of  the  distribution 

are  required  to  exist. 
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and 


Im  {  f(0) }  =  0 


(8) 


umi!! MIS  =  x  (9) 

Equation  (7)  is  the  general  equation  allowing  numerical  evaluation  of  the 
MDF  P(X)  directly  from  the  CF  f(£).  For  a  discontinuous  CDF,  in  order  to 
minimize  inaccuracies  in  a  numerical  evaluation  of  Eq.  (7),  values  of  the  MDF 
P(X)  at  points  removed  from  the  discontinuity  locations  (if  known)  should  be 
computed.  In  particular,  for  a  discrete  random  variable,  values  of  the  MDF 
at  points  midway  between  discontinuities  should  be  computed  when  using! 

Eq.  (7). 

The  integral  in  Eq.  (7)  is  confined  to  the  real  axis.  Since 

|f(0|  <  ydxp(x)  =  l  for  {  real,  (10) 

there  are  no  singular  points  along  the  £  axis.  Also  some  CF's  are  defined 
only  for  €  real;  for  example,  for 

p(x)  -  "7”hi  *  (11) 


f(0  -  exp(-|f|),  real  f,  (12) 

but  f({)  is  not  defined  for  complex  {,  Thus,  the  CF  f(f)  does  not  have  to  be 
analytic  at  fits  origin  to  apply  Eq,  (7),  Nor  do  any  momenta  of  the  random 
variable  have  to  exist. 


DISCRETE  DISTRIBUTIONS 

The  expression  (7)  applies  to  all  MDF1*  (and  CDF's  through  Eq.  (5)); 
however,  it  requires  an  infinite  integral  for  each  value  of  X  Here  we  shall 
alleviate  fids  requirement  for  a  special  class  of  random  variables.  Namely,  we 
consider  discrete  random  variables  that  can  only  take  on  values  which  are 
multiples  of  some  fundamental  Increment  A.  That  is,  the  PDF  of  Interest 
takes  file  form 


Ph)  “  E  °w  *<*-*«£)• 
k  * 


(13) 
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(A  sum  without  limits  is  over  the  integers  from  -«  to  +» . )  Then  the  CF  is 

m  -  £c.  exp(ikAf),  (14) 

k  K 

which  is  periodic  with  period  2*7 A.  Therefore,  the  coefficients  Jc^  |  can  be 
determined  from  the  CF  f(()  by 

ok  =  f;  f  d(  exp(-ikA{)  f(()  ,  (15) 

2t/& 

where  the  integral  is  over  any  interval  of  length  2#/ A. 

Equation  (15)  gives  the  area  of  any  impulse  in  the  PDF  p(x)  in  terms  of 
a  finite  integral  of  the  CF  f(£).  Since  we  are  interested  in  the  CDF  PrCD,  a 
sum  over  |  c^ }  is  required.  At  this  point,  it  is  convenient  to  distinguish  two 
cases:  (1)  nonnegative  discrete  random  variables  and  (2)  general  discrete 
random  variables. 


NONNEGATIVE  DISCRETE  RANDOM  VARIABLES 

If  x  is  a  nonnegative  discrete  random  variable,  the  CDF  is,  at  integer 
value  M, 


Pr(M) 


A 

2  w 


f dt 


exp(>ikAf),  M20, 


where  we  have  substituted  Eq,  (15).  Now 


exp(-ikAl) 


1  -exp  [-IATj 


(18) 


(17) 
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which  must  be  interpreted  as  M  +  1  at  £  =  0,  +2v/A,  +4ir/A .  Using 

Eq.  (17)  and  the  fact  that  f(-£)  =  f*({).  we  note  that  Eq.  (16)  becomes 

Pr(M,  .A  /d(  m  exp  (-'MAWI^gg 
2r/A 


*  T  /  d{  Re  |f«)exp(-lMi(/2)| .  M  >0,  (18) 

where  the  interval  (~v/A,  »/A)  has  been  selected  for  integration.  The  ratio 
of  sines  is  interpreted  as  M  +  1  at  the  origin  {  =  0.  Equation  (18)  is  a 
single  finite  integral  from  which  the  CDF  Pr(M)  can  be  evaluated  at  any  M 
directly  from  the  CF  f((). 

A  special  case  of  Eq.  (18)  is 

»/A 

Pr(0)  -  c o  =  *7  /  dt  Re  |f(f)|  .  (19) 

0 

(Actually,  c0  is  always  given  by  this  formula,  even  for  general  discrete 
random  variables,  as  may  be  seen  from  the  general  formula  (Eq.  (15)). ) 

The  case  of  a  discrete  random  variable  taking  on  values  in  a  semi- 
infinite  range  fi.e, .  KN)  or  (N,«),  where  N  is  finite  but  can  be  positive 
or  negative)  can  be  handled  in  a  similar  fosbion.  The  key  is  that  a  finite  sura  of 
exponentials  (like  Eq,  (17))  can  be  evaluated  without  requiring  a  summation. 


GENERAL  DISCRETE  RANDOM  VARIABLES 


Here  we  shall  consider  discrete  random  variables  which  can  take  on 
values  in  the  range  ( *).  From  Eqs,  (7),  (4),  and  (13)* 

m 

P(0)-|-i>y,^Im{f(|)f  (20) 

0 

1 

°k +2co 

k*-«o 
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That  is,  the  value  of  the  MDF  P (X)  at  the  origin  can  be  evaluated  by  a  single 
infinite  integral.  There  does  not  seem  to  be  any  simpler  way  of  obtaining  this 
number,  which  will  be  necessary  in  the  development  to  follow.  In  some  cases, 
it  may  be  possible  to  evaluate  the  particular  value  P(0)  from  the  integral 
Eq.  (20)  in  closed  form,  or  expand  it  in  a  rapidly  convergent  series,  while 
P(X)  could  not  be  so  evaluated  generally  for  2^  0.  In  any  event,  Eq.  (20)  will 
be  the  only  infinite  integral  necessary  to  evaluate  in  order  to  get  the  complete 
CDF  for  this  general  discrete  case. 

The  area  of  the  impulse  at  the  origin  ie  given  by  Eq.  (15)  as 

*/& 

cq  =~  J  d«Re|f(*)|.  (22) 

0 

Now  let  us  define  auxiliary  functions 

M 


S+(M)  =  ^ 

V 

M> 0  , 

(23) 

0 

S  (M)  «  V 

k.v 

M >0  . 

(24) 

By  a  development  similar  to  Eqs.  (16)  through  (18),  we  find  that  these  auxiliary 
(Unctions  can  be  expressed  directly  in  terms  of  the  CF  f({)  *a 

6+ (M)  Re |f(S exp(+lMAi/2)| ,  M£0,  (25) 

where  the  ratio  of  sines  is  interpreted  as  M  ♦  1  at  the  origin  t  »  0. 

The  CDF  Pr(M)  then  can  be  evaluated  at  any  M  according  to 

P(0) *?o  +  S  (M),  M >0 
2  0  + 

p(°)+jCo*  8jjM+i|),  M<0 


Pr(M) « 
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Here  P(0)  is  given  by  Eq.  -20),  c0  by  Eq.  (22),  and  S+\(M)  by  Eq.  (25),  The 
constants  P(0)  and  c0  need  be  evaluated  once,  but  Eq.-(25)  must  be  evaluated 
for  each  M  of  interest.  However,  Eq.  (25)  is  a  finite  integral. 


EXAMPLES 

We  shall  consider  two  examples  recently  examined  by  Helstrom^  for 
purposes  of  comparison. 

Example  1  -  Exponential  Distribution 


'  3Xp  (-X)v  X>0 

ni‘y\  r  / 

} 

c''-)  V 

1.  0,  x<0 

(27) 

/I  -  exp 

(-X), 

X>°  ) 

Pr(£)  =  P(X>  =< 

0, 

x<o ) , 

(28) 

f<£)  =  (l-i|f 1  . 

(2S) 

The  exact  CDF  is  given  in  Eq.  (28).  Approximate  values  for  the  CDF  are 
obtained  by  substituting  Eq.  (29)  into  Eq.  (7)  and  approximating  the  infinite 
integral  by  a  finite  sum.  Results  are  indicated  in  Table  1. 

The  integral  of  Eq.  (7,  was  sampled  in  {  at  values  indicated  by  column 
four  of  Table  1  and  approximated  by  the  trapezoidal  mle  for  integration.  The 
limit  of  integration  in  Eq.  (7)  was  taken  to  be  the  value  above  60  where  the 
finite  sum  deviated  most  from  the  exact  answer.  Thus,  the  finite  sum  in 
column  three  of  Table  1  is  the  worst  approximation  to  the  exact  answer  in 
column  two, 

F  r  this  example,  the  largest  error  occurred  at  the  origin.  This 
happened  because  the  integrand  of  Eq.  (7)  oscillates  for  X?  0,  thereby  con¬ 
verging  fairly  rapidly,  whereas  the  integrand  decreases  monotonically  only  as 
(1+52)-1  for  X  =  0. 
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Table  1 

NUMERICAL  COMPUTATION  OF  EXPONENTIAL  DISTRIBUTION 


- J 

X 

Pr® 

Finite  Sum 
via  Eq.  (7) 

Increment 
in  $ 

Approximate  Limit 
of  Integration 

-10 

0 

. 00001 

.1 

60 

-2 

0 

-.00007 

.5 

60 

-1 

0 

. 0P008 

.5 

60 

0 

0 

, 00532 

.5 

60 

.2 

. 18127 

.18096 

.5 

60 

1 

.63212 

,63220 

.5 

60 

2 

.86466 

.86470 

.5 

60 

10 

.9999546 

.9999637 

.1 

60 

Example  2  -  Poisson  Distribution 


00  k 

p(x)  *  exp(-A)  V  ^-5(x-k),  (30) 

k=0  k! 


Pr(M)  = 


M  k 

exp(-A)  ^  »  M>0 

k=0 

0,  M <0 


f(()  =  exp[x]exp(i£)-l}]  . 


(31) 

(32) 


The  exact  CDF  is  given  in  Eq.  (31).  Approximate  values  for  the  CDF  are 
obtained  by  substituting  Eq.  (32)  into  Eq,  (18),  with  A  =  1,  and  approximating 
the  finite  integral  by  a  finite  sum.  Results  are  indicated  in  Table  2. 
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Table  2 

NUMERICAL  COMPUTATION  OF  POISSON  DISTRIBUTION 


M 

Pr(M) 

Finite  Sum  via  Eq.  (18) 

Number  of 
Intervals 

0 

.00000  03059 

.00000  03059 

25 

1 

.00000  48944 

.00000  48945 

25 

6 

.00763  18996 

.00763  18998 

25 

14 

.46565  37089 

.46565  37089 

25 

16 

.66412  32005 

.66412  32004 

25 

20 

.91702  90899 

.91702  90895 

25 

29 

.99958  15502 

.99958  15500 

25 

30 

.99980  26867 

.99980  26865 

25 

40 

.99999  99765 

.99999  99764 

25 

The  integral  of  Eq.  (18)  was  divided  into  25  equal  intervals  and  approxi¬ 
mated  by  the  trapezoidal  rule  for  integration.  Columns  two  and  three  of  Table  2 
show  that  the  error  in  the  approximation  occurs  in  the  tenth  place  (and  may  be 
due  to  computer  inaccuracies  rather  than  sampling  errors).  Also,  the 
accuracy  holds  on  the  tails  of  the  CDF  as  well  as  near  the  mean. 


CONCLUSIONS 

The  numerical  technique  suggested  for  obtaining  CDF's  directly  from 
CF's  has  considerable  merit.  It  requires  no  moment  evaluations  or  series 
expansions  (like  Edgeworth  or  Laguerre)  for  the  distributions.  It  does  not  de¬ 
pend  upon  evaluation  of  derivatives  of  CF's,  but  depends  only  upon  the  values  of 
the  CF  itself.  (Evaluation  of  high-order  derivatives  can  be  extremely  tedious 
and  time-consuming  even  if  an  analytic  form  for  the  CF  is  available. )  The 
accuracy  of  the  suggested  technique  can  be  estimated  and  controlled  by  de¬ 
creasing  the  increment  in  foe  integral  evaluations  or  lengthening  the  interval  of 


•  t 
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integration  or  both;  the  change  in  the  approximation  la  a  measure  of  the  error 
at  that  point.  The  method  does  not  require  an  inordinate  number  of  samples  of 
the  CFt  at  least  for  the  examples  considered,  and  the  additional  functions 
requiring  evaluation  are  sines  and  cosines.  Intermediate  evaluation  of  the 
PDF  is  entirely  circumvented.  (Of  course,  estimates  of  the  PDF  are  avail¬ 
able  as  differences  of  the  CDF,  if  desired. ) 
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